Domain Decomposition for Stochastic Optimal Control

This work proposes a method for solving linear stochastic optimal control (SOC) problems using sum of squares and semidefinite programming. Previous work had used polynomial optimization to approximate the value function, requiring a high polynomial degree to capture local phenomena. To improve the scalability of the method to problems of interest, a domain decomposition scheme is presented. By using local approximations, lower degree polynomials become sufficient, and both local and global properties of the value function are captured. The domain of the problem is split into a non-overlapping partition, with added constraints ensuring $C^1$ continuity. The Alternating Direction Method of Multipliers (ADMM) is used to optimize over each domain in parallel and ensure convergence on the boundaries of the partitions. This results in improved conditioning of the problem and allows for much larger and more complex problems to be addressed with improved performance.Comment: 8 pages. Accepted to CDC 201

Analysis of Control Systems on Symmetric Cones

It is well known that exploiting special structure is a powerful way to extend the reach of current optimization tools to higher dimensions. While many linear control systems can be treated satisfactorily with linear matrix inequalities (LMI) and semidefinite programming (SDP), practical considerations can still restrict scalability of general methods. Thus, we wish to work with high dimensional systems without explicitly forming SDPs. To that end, we exploit a particular kind of structure in the dynamics matrix, paving the way for a more efficient treatment of a certain class of linear systems. We show how second order cone programming (SOCP) can be used instead of SDP to find Lyapunov functions that certify stability. This framework reduces to a famous linear program (LP) when the system is internally positive, and to a semidefinite program (SDP) when the system has no special structure

Robustness, Adaptation, and Learning in Optimal Control

Recent technological advances have opened the door to a wide variety of dynamic control applications, which are enabled by increasing computational power in ever smaller devices. These advances are backed by reliable optimization algorithms that allow specification, synthesis, and embedded implementation of sophisticated learning-based controllers. However, as control systems become more pervasive, dynamic, and complex, the control algorithms governing them become more complex to design and analyze. In many cases, optimal control policies are practically impossible to determine unless the state dimension is small, or the dynamics are simple. Thus, in order to make implementation progress, the control designer must specialize to suboptimal architectures and approximate control. The major engineering challenge in the upcoming decades will be how to cope with the complexity of designing implementable control architectures for these smart systems while certifying their safety, robustness, and performance. This thesis tackles the design and verification complexity by carefully employing tractable lower and upper bounds on the Lyapunov function, while making connections to robust control, formal synthesis, and machine learning. Specifically, optimization-based upper bounds are used to specify robust controllers, while lower bounds are used to obtain performance bounds and to synthesize approximately optimal policies. Implementation of these bounds depends critically on carrying out learning and optimization in the loop. Examples in aerospace, formal methods, hybrid systems, and networked adaptive systems are given, and novel sources of identifiability and persistence of excitation are discussed.</p

Collaborative System Identification via Parameter Consensus

Classical schemes in system identification and adaptive control often rely on persistence of excitation to guarantee parameter convergence, which may be difficult to achieve with a single agent and a single input. Inspired by consensus systems, we extend classical parameter adaptation to the multi agent setting by combining an adaptive gradient law with consensus dynamics. The gradient law represents the main learning signal, while consensus dynamics attract each agent's parameter estimates toward those of its neighbors. We show that the resulting decentralized online parameter estimator can be used to identify the true parameters of all agents, even if no single agent employs a persistently exciting input

Automata theory meets approximate dynamic programming: Optimal control with temporal logic constraints

We investigate the synthesis of optimal controllers for continuous-time and continuous-state systems under temporal logic specifications. The specification is expressed as a deterministic, finite automaton (the specification automaton) with transition costs, and the optimal system behavior is captured by a cost function that is integrated over time. We construct a dynamic programming problem over the product of the underlying continuous-time, continuous-state system and the discrete specification automaton. To solve this dynamic program, we propose controller synthesis algorithms based on approximate dynamic programming (ADP) for both linear and nonlinear systems under temporal logic constraints. We argue that ADP allows treating the synthesis problem directly, without forming expensive discrete abstractions. We show that, for linear systems under co-safe temporal logic constraints, the ADP solution reduces to a single semidefinite program

Automata theory meets approximate dynamic programming: Optimal control with temporal logic constraints

We investigate the synthesis of optimal controllers for continuous-time and continuous-state systems under temporal logic specifications. The specification is expressed as a deterministic, finite automaton (the specification automaton) with transition costs, and the optimal system behavior is captured by a cost function that is integrated over time. We construct a dynamic programming problem over the product of the underlying continuous-time, continuous-state system and the discrete specification automaton. To solve this dynamic program, we propose controller synthesis algorithms based on approximate dynamic programming (ADP) for both linear and nonlinear systems under temporal logic constraints. We argue that ADP allows treating the synthesis problem directly, without forming expensive discrete abstractions. We show that, for linear systems under co-safe temporal logic constraints, the ADP solution reduces to a single semidefinite program

Constrained autonomous satellite docking via differential flatness and model predictive control

We investigate trajectory generation algorithms that allow a satellite to autonomously rendezvous and dock with a target satellite to perform maintenance tasks, or transport the target satellite to a new operational location. We propose different path planning strategies for each of the phases of rendezvous. In the first phase, the satellite navigates to a point in the Line of Sight (LOS) region of the target satellite. We show that the satellite's equations of motion are differentially flat in the relative coordinates, hence the rendezvous trajectory can be found efficiently in the flat output space without a need to integrate the full nonlinear dynamics. In the second phase, we use model predictive control (MPC) with linearized dynamics to navigate the spacecraft to the final docking location within a constrained approach envelope. We demonstrate feasibility of this study by simulating a sample docking mission